**BREAKING** pygmt.grdcut: Refactor to store output in virtualfiles for grids #3115
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 tasks
Minimal xarray.DataArray output with data and coordinates, no metadata yet.
weiji14
reviewed
Mar 22, 2024
seisman
changed the title
seisman
force-pushed
the
datatypes/gmtgrid
branch
3 times, most recently
from
d8f1160 to
d0517e0
Compare
seisman
force-pushed
the
datatypes/gmtgrid
branch
from
d0517e0 to
4ba67df
Compare
31 tasks
weiji14
added a commit
that referenced
this pull request
Jun 18, 2024
Special case `earth_day` to be loaded as GMT_IMAGE rather than GMT_GRID. Need to use `GMT_OUT|GMT_IS_REFERENCE` in virtualfile_out to avoid segfault, xref #3115.
Extra metadata from the _GMT_GRID_HEADER struct.
Reorder the dimensions to follow Channel, Height, Width (CHW) convention. Also added doctest checking output DataArray object and the image's x and y coordinates.
Get the registration and gtype info from the grid header and apply it to the GMT accessor attributes.
Trying to match some of the doctests in _GMT_GRID.
seisman
commented
Sep 29, 2024
pygmt/clib/session.py
Outdated
Comment on lines
2332
to
2334
| # The logic here is because: an image can be read into a grid container, but a grid | ||
| # can't be read into an image container. So, try to read the file as an image first. | ||
| # If fails, try to read it as a grid. |
There was a problem hiding this comment.
It turns out the logic here is not 100% correct.
- An image can be read into a grid container, but only the first band is read.
- A grid can be read into an image container, as long as another libgdal package (maybe
libgdal-netcdforlibgdal-hdf5) is installed. xref: Add libgdal-hdf5 dependency conda-forge/gmt-feedstock#293
There was a problem hiding this comment.
- A grid can be read into an image container, as long as another libgdal package (maybe
libgdal-netcdforlibgdal-hdf5) is installed. xref:
The first try-except check is to try reading into a GMT_IMAGE struct, and return "image" if there are 3 bands, and "grid" if there is only 1 band. Even if grids can be read into GMT_IMAGE, we will still call it a grid if it has 1-band right?
There was a problem hiding this comment.
Trying to refresh my memory about what's happening:
- Reading a grid into
GMT_GRID: Works! - Reading an image into
GMT_GRID: Only the first band is read. - Reading a grid into
GMT_IMAGE: It depends onlibgdal-hdf5being installed. Read as an single-band image if installed, otherwise raise errors. - Reading an image into
GMT_IMAGE: works.
import pygmt
pygmt.grdcut("@earth_relief_01d_p", region=[0, 10, 0, 10])
If libgdal-hdf5 is not installed, _raster_kind first read it as an image, which fails and reports following errors:
ERROR 4: `/home/runner/.gmt/server/earth/earth_relief/earth_relief_01d_p.grd' not recognized as being in a supported file format. It could have been recognized by driver HDF5, but plugin gdal_HDF5.so is not available in your installation. You may install it with 'conda install -c conda-forge libgdal-hdf5'
Error: ession [ERROR]: Unable to open earth_relief_01d_p.grd.
Error: ession [ERROR]: ERROR reading image with gdalread.
pygmt-session (gmtapi_import_image): Could not read from file [earth_relief_01d_p.grd]
[Session pygmt-session (43)]: Error returned from GMT API: GMT_IMAGE_READ_ERROR (22)
[Session pygmt-session (43)]: Error returned from GMT API: GMT_IMAGE_READ_ERROR (22)
then it tries to read it as a grid, which succeeds. But we still see the error messages above. I believe we need to find a way to suppress these errors or find an alternative way to determine the data kind.
weiji14
reviewed
Sep 30, 2024
seisman
removed
run/benchmark
7 tasks
seisman
commented
Dec 9, 2024
| outkind = kind | ||
| case "_": | ||
| msg = f"Unsupported data type {type(grid)}." | ||
| raise GMTInvalidInput(msg) |
There was a problem hiding this comment.
Lines 117-118 should be caught by the test_grdcut_fails test.
seisman
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR applies virtualfiles in
pygmt.grdcut, so that this function doesn't have to write to temporary files (#2730).grdcutis on of the complicated module that it works for grid, image and cube. We have to know thekindof the input before calling theSession.virtualfile_outmethod.It turns out it's not trivial to determine the data
kindof an input file. We have some options:.ncis a grid,.tifis a image)kindparameter (default to"grid") and users should specify thekindparameter.Option 1-2 may not always work; option 3 also dosn't always work (see #3115 (comment)).
This PR currently adopts option 4. Need to note option 4 meaning BREAKING:
grdcut("@earth_day_01d")works; now must usegrdcut("@earth_day_01d", kind="image")